Multi-class classification via proximal mirror descent

We consider the problem of multi-class classification and a stochastic optimization approach to it. The idea is to, instead of weighing classes, make use of the total sum of margins as a regularization. As general the problem is hard to solve, we use Bregman divergence as the regularizer and end up with a proximal mirror descent with a specific distance-generating function. The approach is designed for problems with highly unbalanced classes as it makes use of different margins between each class and the rest, therefore emulating the one-vs-all approach. This should decrease the error dependence in terms of the number of classes.